We also give a training algorithm usable for all mixed integer linear programs, vastly generalizing the applicability of the framework.

A common lens to theoretically study neural net architectures is to analyze the functions they can approximate.

We also give a training algorithm usable for all mixed integer linear programs, vastly generalizing the applicability of the framework.

A common lens to theoretically study neural net architectures is to analyze the functions they can approximate.

Finding accurate solutions to partial differential equations (PDEs) is a crucial task in all scientific and engineering disciplines. It has recently been shown that machine learning methods can improve the solution accuracy by correcting for effects not captured by the discretized PDE.

To alleviate the annotation burden in supervised learning, N-tuples learning has recently emerged as a powerful weakly-supervised method. While existing N-tuples learning approaches extend pairwise learning to higher-order comparisons and accommodate various real-world scenarios, they often rely on task-specific designs and lack a unified theoretical foundation. In this paper, we propose a general N-tuples learning framework based on empirical risk minimization, which systematically integrates pointwise unlabeled data to enhance learning performance. This paper first unifies the data generation processes of N-tuples and pointwise unlabeled data under a shared probabilistic formulation. Based on this unified view, we derive an unbiased empirical risk estimator that generalizes a broad class of existing N-tuples models. We further establish a generalization error bound for theoretical support. To demonstrate the flexibility of the framework, we instantiate it in four representative weakly supervised scenarios, each recoverable as a special case of our general model. Additionally, to address overfitting issues arising from negative risk terms, we adopt correction functions to adjust the empirical risk. Extensive experiments on benchmark datasets validate the effectiveness of the proposed framework and demonstrate that leveraging pointwise unlabeled data consistently improves generalization across various N-tuples learning tasks.

Evolutional deep neural networks (EDNN) solve partial differential equations (PDEs) by marching the network representation of the solution fields, using the governing equations. Use of a single network to solve coupled PDEs on large domains requires a large number of network parameters and incurs a significant computational cost. We introduce coupled EDNN (C-EDNN) to solve systems of PDEs by using independent networks for each state variable, which are only coupled through the governing equations. We also introduce distributed EDNN (D-EDNN) by spatially partitioning the global domain into several elements and assigning individual EDNNs to each element to solve the local evolution of the PDE. The networks then exchange the solution and fluxes at their interfaces, similar to flux-reconstruction methods, and ensure that the PDE dynamics are accurately preserved between neighboring elements. Together C-EDNN and D-EDNN form the general class of Multi-EDNN methods. We demonstrate these methods with aid of canonical problems including linear advection, the heat equation, and the compressible Navier-Stokes equations in Couette and Taylor-Green flows.

As synthetic data becomes higher quality and proliferates on the internet, machine learning models are increasingly trained on a mix of human- and machine-generated data. Despite the successful stories of using synthetic data for representation learning, using synthetic data for generative model training creates "self-consuming loops" which may lead to training instability or even collapse, unless certain conditions are met. Our paper aims to stabilize self-consuming generative model training. Our theoretical results demonstrate that by introducing an idealized correction function, which maps a data point to be more likely under the true data distribution, self-consuming loops can be made exponentially more stable. We then propose self-correction functions, which rely on expert knowledge (e.g. the laws of physics programmed in a simulator), and aim to approximate the idealized corrector automatically and at scale. We empirically validate the effectiveness of self-correcting self-consuming loops on the challenging human motion synthesis task, and observe that it successfully avoids model collapse, even when the ratio of synthetic data to real data is as high as 100%.